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This study proposes a method based on Dempster-Shafer theory (DST) and fuzzy neural network (FNN) to improve the reliability 
of recognizing fatigue driving. This method measures driving states using multifeature fusion. First, FNN is introduced to obtain the 
basic probability assignment (BPA) of each piece of evidence given the lack of a general solution to the definition of BPA function. 
Second, a modified algorithm that revises conflict evidence is proposed to reduce unreasonable fusion results when unreliable 
information exists. Finally, the recognition result is given according to the combination of revised evidence based on Dempster’s 
rule. Experiment results demonstrate that the recognition method proposed in this paper can obtain reasonable results with the 
combination of information given by multiple features. The proposed method can also effectively and accurately describe driving 


states. 


1. Introduction 


Fatigue is a common physiological phenomenon that reduces 
a driver's attention and ability to control the vehicle. Fatigue 
driving is one of the major causes of road accidents and poses 
a significant threat to the safety of drivers and passengers. 
Common methods for detecting fatigue driving include 
measurements of physiological features, facial features, and 
features of driving behavior [1]. 


Physiological signals related to fatigue consist of elec- 
troencephalogram (EEG), electrocardiogram (ECG), elec- 
tromyography (EMG), and heart rate variability (HRV) [2- 
5]. The results of fatigue recognition by physiological signals 
have high accuracy, but this approach has limitations. One 
particular limitation is the extraction of signals that are intru- 
sive for drivers thereby making them uncomfortable. Non- 
contact measurements, such as extracting facial features and 
features of driving behavior, are practical and will not affect 
normal driving. Existing studies have shown that fatigued 
people exhibit certain visual changes in facial features, such as 
eye closure, movements in gaze direction, yawning, and head 
movements [6-11]. Fatigue may be also reflected in features 


of driving behavior, such as lane departure and steering wheel 
movements [12-14]. However, using facial features to evaluate 
driving states is not always reliable because extraction of facial 
features may be affected by variation of illumination and 
driver's posture. Thus, image processing algorithms cannot 
ensure the accuracy of recognition results. The reliability of 
methods based on driving behavior is dependent on road and 
climate conditions, vehicle types, and driving habits. Thus, 
accurate evaluation of driving states is difficult to achieve 
with a single feature under a complex environment. Models 
based on information fusion have been developed to increase 
the accuracy of fatigue recognition [15-17]. Lee and Chung 
[15] proposed a dynamic fatigue monitoring system based 
on Bayesian network. Deng et al. [17] proposed a fatigue 
monitoring method based on Dempster-Shafer theory (DST). 
However, the prior probability needed in Bayesian network 
is acquired according to experts’ subjective experience. The 
definition of basic probability assignment (BPA) function 
in DST has no standard solution because of the nonlinear 
characteristics of fatigue features. 

DST [18, 19] is an improvement of Bayesian infer- 
ence and an effective method for handling imprecise and 
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FIGURE 1: Frames of facial expression video. 


uncertain information; DST has been widely used in infor- 
mation fusion. However, Dempster’s rule of combination 
often obtains unreasonable fusion results when proofs are 
in conflict [20]. Generally speaking, there are two main 
reasons to cause highly conflicting evidence [21]. One is the 
questionable information reliability caused by environmental 
disturbance or instrument errors. To solve this problem, 
a series of improvement methods have been proposed. 
These methods are mostly focused on the modifications of 
combination rules and revisions of original evidence. The 
modifications of combination rules include Yager’s rule [22], 
Qian et al’s rule [23], Lefevre et al’s rule [24], and Dezert- 
Smarandache theory (DSmT) [25]. The revisions of original 
evidence consist of weighted average strategy [26, 27] and 
discount strategy [28-31]. Jiang et al. [32] used Z-numbers 
[33] to evaluate the fuzziness and reliability of the uncertainty 
in sensor data fusion. The other reason is that the given 
environment is in an open world, which means the frame of 
discernment is incomplete due to lack of knowledge [34]. The 
information fusion environment of traditional DST is in a 
closed world, which means the frame of discernment consists 
of all the elements. Deng [21] proposed the generalized 
evidence theory (GET) to deal with uncertain information 
fusion in the open world. Jiang et al. [35] measured the weight 
of evidence based on Deng entropy [36] to handle conflict. 

This study proposed a method based on DST and fuzzy 
neural network (FNN) [37] to recognize fatigue driving. 
Facial features were adopted as the fatigue parameters of 
the proposed method, which were extracted by cameras 
mounted in the vehicle. Given the self-learning and self- 
adaption abilities of FNN [38], this method was used to 
obtain the BPA of evidence. Under the driving environment, 
conflict information was mainly caused by environmental 
interferences and measurement errors. To address this issue, a 
modified algorithm is introduced into the method by taking 
the credibility of each piece of evidence into consideration. 
This algorithm revises original evidence based on discount 
strategy to improve the reliability of recognition result. 

This study is organized as follows. Measurements of 
fatigue features are summarized in Section 2. Section 3 pro- 
poses a recognition method characterized by multifeature 
fusion. Section 4 discusses the experiments and results. Con- 
clusions are given in Section 5. 


TABLE 1: The evaluation criteria of drivers mental states. 


State Score Features 


Eyes open properly and close 

intermittently and rapidly, active 
eyeball movement, head straight, 
glance sideways occasionally, etc. 


Awake 0 


Eye closure trend appears, eye 
activity decreases, yawning, 
squeezing eyes, nod subconsciously, 
etc. 


Fatigue 1 


Long time eyes closed, hard to keep 
eyes open, blank stare, slow eyeball 
movement, head skew, etc. 


Severe fatigue 2 


2. Measurements of Fatigue Features 


2.1. States Evaluation Criteria. The change in driver’s mental 
states is a continuous process. Driver fatigue is characterized 
by various facial features, such as droopy eyelid, eye closure, 
change in gaze direction, yawning, and change in head 
posture. Using the camera placed on the dashboard, in front 
of the driver, we can capture the facial expression video. 
Frames in the video that represent several expressions are 
shown in Figure 1. 

Kimura et al. [39] analyzed the feasibility of using facial 
expressions to judge the degree of fatigue and the consistency 
of results between the facial features analysis and ratings of 
facial expression videos. In this paper, we quantify the driver's 
states into three levels and give corresponding scores for each 
level: Awake (0 point), Fatigue (1 point), Severe fatigue (2 
points). The evaluation criteria of driver's states are shown in 
Table 1. 

Under the vehicle interior environment, extracting fea- 
tures accurately and rapidly becomes difficult because of lim- 
itations in image processing algorithms and environmental 
interferences. To address this issue, this study considers visual 
features that can be effectively extracted and measured, like 
eye movement and mouth movement. 


2.2. Facial Features Extraction. In order to obtain the eye 
movement and the moth movement features, we need to 
locate eyes and mouth areas first. In this paper, CLM 
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FIGURE 2: Detection results of CLM algorithm. 








FIGURE 3: Eyes and mouth areas. 
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FIGURE 4: Eyelid movement. 


(Constrained Local Model) [40] algorithm is adopted to 
localize facial landmarks. The facial feature points detected 
from several frames are shown in Figure 2. 


Eyes and mouth areas can be extracted according to the 
coordinates of feature points, as are shown in Figure 3. By 
analyzing the relation between feature points, we can acquire 
some states evaluation indexes. 

Detecting fatigue through eye features, we mainly focus 
on the analysis of eyelid and iris movement. Eyelid move- 
ment, reflected by eyes opening and closing, is one of the 
most relevant and also the most visually significant fatigue 
features. We use changes in distance between upper and lower 
eyelids over a period of time to represent eyelid movement. 
Figure 4 shows the eyelid movement in a period of time. 

As is shown in Figure 4, T is the width of time window 
and L is the average distance between eyelids. According 
to PERCLOS (Percentage of Eyelid Closure) [41], we take 
approximate 0.7L as the threshold to determine whether 
the eye is opened, which is denoted with the broken line 





FIGURE 5: Iris features. 


in the figure. Thus, we can get the number of blinks, 
denoted as N,, and each eye closing duration, denoted as 
t; (i = 1,2,...,N,) within a given time window. We choose 
PERCLOS (Percentage of Eyelid Closure) and MCD (Max- 
imum Closing Duration) as the eyelid movement features 
to measure fatigue. PERCLOS and MCD are calculated as 
follows: 


i 
PERCLOS = = ) t; 
T (1) 


MCD S mox hides N 


Iris movement is mainly reflected in the change of driver's 
gaze direction. To get iris movement features, we use Canny 
algorithm [42] and Hough transform [43] to locate iris region 
and get the center of iris, as is shown in Figure 5. 


In Figure 5, L, and R, are left and right corner of eye at 
time t. O, is the center of iris at time t, and O,,, is the center 
of iris at time t + 1. The width of time window is T, and the 
pupil rest time is t; (1 < i < N). Total number of frames 
within the time window is N. Features extracted according 
to iris movement are AAI (Average Asymmetry of Iris) and 





FIGURE 6: Measurement of width and height of the mouth. 


PRPT (Percentage of Pupil Rest Time). AAI and PRPT are 
calculated as follows: 


m-i) 





N (2) 
1 << t-+1, |O,0,,,| 28€, 
PRPT=—)¢t, t= iVin1 
Nf ia t; otherwise, 


where | - | represents the distance between two points and ¢ is 
the threshold of iris movement. 

Yawning is also one of the most visually significant fatigue 
features. The width and height of mouth can be calculated 
according to the feature point coordinates in the mouth 
region, as is shown in Figure 6. And we use the aspect ratio 
to measure the mouth opening, defined as k = H,,,/W,,. 

Figure 7 shows the mouth movement reflected by changes 
in mouth opening. T is the width of time window. When k > 
0.7, it means that mouth is opened. Yawning can be consid- 
ered as a process that mouth keeps opening more than three 
seconds. The number of yawning is denoted as N,, and each 
mouth open duration is denoted as t; (i = 1,2,...,N i) 

The fatigue indexes extracted from Figure 7 are YF 
(Yawning Frequency) and AOT (Average Opening Time). 
They can be calculated as follows: 


N 
YF = —, 
T 
n (3) 
| y 
Ny i=] 


3. Framework of Multifeature 
Fusion Recognition 


The recognition method proposed in this paper combines the 
advantages of DST and FNN. A single FNN is divided into 
several subnetworks. Each subnetwork with nonlinear map- 
ping capability is trained to process information from dif- 
ferent features to conduct preliminary evaluation of driving 
states. The BPA function is defined by normalizing network 
output. The problem of highly conflicting evidence fusion can 
be solved efficiently by revising conflicting evidence using 
discount strategy. The degree of conflict is measured by the 
correlation coefficient of evidence to calculate the credibility 
of evidence, which is used as discount factor. Evidence 
fusion is based on DST. Figure 8 shows the framework of 
multifeature fusion fatigue recognition. 
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FIGURE 7: Mouth movement. 


3.1, DST. Let © = {0,,0,,...,0,,} be a finite set of mutually 
exclusive and exhaustive proposition known as the frame of 
discernment. Power set 2° is the set that includes all subsets 
of ©. BPA is a function defined as m : 2° — [0,1], which 
satisfies the following condition: 


> m(A) = 1, 
AcO (4) 


m(¢$) = 0. 


The value of m(A) represents the degree of evidential 
support of exact set A. If m(A) > 0, subset A is called focal 
element. 

Let m, and m, be two BPAs defined on frame ©, which 
are derived from two independent sources. Focal elements 
are B; (i = 1,2,...,m) and C; (j = 1,2,...,n). Combined 
BPA can be calculated according to Dempsters rule of 
combination: 


m (A) 


E DBAC)=A Z sL (Ci) hee Aes (5) 


0, A= @, 


where conflicting factor k = ŽB nC =ø m,(B;)m,(C A This 
equation measures the conflict degree between m, and m,, 
and 1/(1 — k) represents the normalization factor, which is 
used to avoid assigning nonzero probability to the empty set. 


3.2. Determination of BPA. In multifeature fusion recogni- 
tion, each feature is considered as a piece of evidence. Pieces 
of evidence are combined according to Dempster's rule and 
new evidence is obtained as basis for recognition. In practical 
applications, the definition of BPA function is based on the 
characteristics of data. Common approaches include experi- 
mental formula, neural network method, fuzzy set method, 
and gray correlation analysis [44-47]. Neural network can 
conduct generalization, which can play the role of experts in 
determining BPA after learning a certain number of training 
samples. In this study, BPAs are constructed by normalizing 
the output of FNN. BPA function is defined as follows: 
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where A; stands for driving state and y(A;) corresponds to 
the ith node of output. 


m(A;,) = om ee eee 2 (6) 


3.2.1. Structure of FNN. In order to get driving states from 
eyelid movement, iris movement, and mouth movement 
features, we design three FNNs with two inputs and three 
outputs to present signs of fatigue. As is shown in Table 1, 
driving states are classified into three levels: Awake, Fatigue, 
and Severe fatigue. So the FNN outputs corresponded to 
these three levels. For eyelid movement measurement, the 
inputs of FNN are PERCLOS and MCD. For iris movement 
measurement, the inputs of FNN are AAI and PRPT. For 
mouth movement measurement, the inputs of FNN are YF 
and AOT. Structure of FNN is shown in Figure 9. When a 
state is identified, the corresponding output node is set to 1; 
otherwise, it is set to 0. Normalizing the FNN output data over 
a period of time and BPA of each piece of evidence can be 
obtained by formula (6). 

FNN is composed of five layers, namely, input, fuzzy, 
fuzzy rules, normalized, and output layer. 

The first layer is input layer. Each node in this layer 
represents an input variable x,. 

The second layer is fuzzy layer. Each node represents the 
value of a linguistic variable. The fuzzy layer calculates the 
subjection function of each input variable that belongs to a 
fuzzy set that corresponds to the value of a certain linguistic 
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FIGURE 10: Curve of subjection function. 


variable. For all the features defined in Section 2, their values 
increase along with the accumulation of fatigue. Therefore, we 
classify the input into three different linguistic variable values, 
namely, small, medium, and large. The subjection functions 
of different linguistic variables are assigned as follows: 


l 
l + exp lo; (<x; — c) ' 
(x-a) 


2 > 
Oi (7) 


| l 
large: u! (x;) = —— 
arge Mi (x<) 1 + exp [-0; (2 — aj) 


small: u’ (x;) = 


medium: u’ (x;) = exp | — 


? 


b= l 2sm = ly Zysxes iy 

The subjection function of linguistic variable medium is 
described by a Gaussian function, where c,, and g;; represent 
the central value and width of the function that belongs to 
jth fuzzy set of ith input variable. The subjection functions of 
linguistic variables small and large are described by Sigmoid 
function, where œ; contributes to the right shift of the 
subjection function along the horizontal axis and o;; adjusts 
the shape of function. n is the number of input variables and 
m; is the fuzzy partitions of x;. The number of nodes in this 
layer is N,, N; = Di, M; 

The outputs of these functions are normalized to range 
from 0 to 1. The curve of subjection function based on 
formula (7) is shown in Figure 10. 


(Severe fatigue) 


(Severe fatigue) 


6 
TABLE 2: Fuzzy rules of eyelid movement. 

xy Xa Ji J3 Y3 
(PERCLOS) (MCD) (Awake) (Fatigue) 

Small Small l 0 0 
Small Medium l 0 0 
Medium Small l 0 0 
Medium Medium l 0 0 
Medium Large 0 l 0 
Large Medium 0 1l 0 
Large Large 0 0 1 

TABLE 3: Fuzzy rules of iris movement. 

xy Ag Xi J2 Y3 
(AAI) (PRPT) (Awake) (Fatigue) 

Small Small l 0 0 
Small Medium l 0 0 
Medium Small l 0 0 
Medium Medium l 0 0 
Medium Large 0 l 0 
Large Medium 0 l 0 
Large Large 0 0 1 


The third layer is fuzzy rules layer. Each node represents 
one fuzzy rule. By calculating the subjection degree, the 
fitness of each rule can be defined as 


where i, € {1,2,... 
..,M,}, j = 1,2,...,m,m = [];_,m,, and m is the number 
of nodes in this layer, which is equal to the number of fuzzy 


rules. 


Mi ts te € {hess 


= iy ty : in 
OO; = Mi M ``’ Mn? 


stil fress 


(8) 


s1 E12 


According to the facial expression of fatigue, take eyelid 
features as an example, the fuzzy rules can be described as 


follows: 


If PERCLOS is small, MCD is small, then output is 
Awake. 


If PERCLOS is medium, MCD is large, the output is 
Fatigue. 


If PERCLOS is large, MCD is large, then output is 


Severe fatigue. 


The fuzzy rules of eyelid movement, iris movement, and 
mouth movement are shown in Tables 2, 3 and 4. 
The fourth layer is normalized layer. The normalized 


calculation is defined as follows: 


a; = 


an »? 
pe a 


ai 


T= Laa 


(9) 
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TABLE 4: Fuzzy rules of mouth movement. 


Xı Aa Yı Ja Y3 
(YF) (AOT) (Awake) (Fatigue) (Severe fatigue) 
Small Small 1 0 0 
Small Medium 1 0 0 
Medium Small 1 0 0 
Medium Medium 0 1 0 
Medium Large 0 l 0 
Large Medium 0 l 0 
Large Large 0 0 1 


The fifth layer is output layer, which is also called 
defuzzification layer. Each node in this layer represents an 
output variable y;. The defuzzification is defined as follows: 


m 
Yi = > wij 1 = 1, PREE o (10) 
j=1 


where w;; stands for the weight of FNN, which can be adjusted 
through the learning algorithm, and r is the number of output 
variables. 


3.2.2. Learning Algorithm. The error of FNN is defined as 
follows: 


l 
e=>(%- vi)» (11) 
where y; represents the actual output and y, represents the 
expected output. 

Error back propagation algorithm (BP algorithm) is used 
for network parameter adjustment to ensure that the actual 
output is close to the expected output. Based on BP algorithm, 
the weight of FNN w;; can be adjusted as follows: 





e 
Wij (f+ 1) = Wij (= "30, ; 
(12) 
Oe Oe OY; _ 
te ee 
Ow; Oy; Ow; ; l 


And the subjection function parameters c; and 0; can be 
adjusted as follows: 


de 
n (13) 
e 
Oi; (t +1) = Ojj g= 1oy 


where y stands for the learning rate, 4 > 0. 


3.3. Revision of Evidence. Given the limitation of DST, unrea- 
sonable results are often obtained when combining highly 
conflicting evidence. Errors in the feature parameters 
extracted by camera are inevitable while driving because of 
environmental interferences. Therefore, the Dempster’s rule 
cannot be used directly when conflict exists. 
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Evidence with low reliability should not be negated com- 
pletely because the cause of the conflict remains unknown. 
Thus, in the modified algorithm of recognition method, the 
original pieces of evidence are revised through rational distri- 
bution of unreliable evidence before performing information 
fusion based on DST. 


The discount strategy proposed by Shafer [19] is applied 
in this method. The credibility of evidence, which is regarded 
as the discount factor, is used to revise the BPAs of original 
evidence. Parts of the unreliable credibility are distributed to 
set © according to the discount rule. The influence on the 
result of fusion of conflicting evidence can be reduced by 
increasing the uncertainty of evidence. 


Discount rule is defined as follows: 


m; (A) = OM; (A) , A#O, 


(14) 
m; (©) =1- ) m, (A). 

The key of discount strategy is to measure the credibility 
of evidence effectively. Conflicting factor k in DST is used 
to measure the degree of conflict. The larger the value of k, 
the higher the degree of conflict. However, when pieces of 
evidence are highly conflicting, k — 1. This result means that 
conflicting factor k cannot accurately represent the degree of 
conflict. 


The correlation coefficient of evidence is proposed in this 
paper to measure the credibility of evidence. 

Let E, and E, be two pieces of evidence on frame ©; 
BPAs are m, and m,; and focal elements are A; and B,. The 
correlation coefficient between E} and E, is defined as 


VÈ rå (4) (Z rå (B;)) 


where sim(m,,m,) € [0,1]. The larger the value of 
sim (m,,m,), the higher the degree of correlation between 
pieces of evidence. 


sim (mpi) = 


(15) 


For n pieces of evidence provided by multiple features, 
the correlation coefficient can be expressed as the following 
correlation matrix: 


S 
1 sim (m, m) n sim (m, m,,) 
sim (m,, m) l - sim (m,,m,,) (16) 
sim(m,,m,) sim (m,m) = l 


Pieces of evidence with high correlation coefficient can 
support each other. Therefore, the degree of one piece of 
evidence supported by others can be defined based on the 
correlation matrix as follows: 





sup (m;) = : > sim (m;,m,), (17) 


where support degree sup (m,) represents the reliability of 
evidence. The evidence with the highest degree of support 
is used as standard evidence. The weight of each piece 
of evidence compared with the standard evidence can be 
calculated and considered as the credibility of evidence: 


maX] <j<y {sup (m;)} 


The credibility of evidence a; is taken as discount factor. 
The original pieces of evidence can then be revised according 
to the discount rule in formula (14). Thus, the reasonability of 
the results increases after combining pieces of evidence using 
Dempster's rule. 


3.4. Recognition of Driving States. Driving state is identified 
based on the BPA of combined evidence. Assuming that 


A,,A, C O, A, and A, are defined as follows: 


m(A,) = max {m (A,), A; c O}, 


(19) 
m(A,)=max{m(A,), A; CO, A; # Aj}. 
If A, satisfies the following recognition rule: 
m(A,)-m(A,)>&, 
m(A,)>m(@), (20) 


m(A;) > & 


where €, and €, are two thresholds, then A, is regarded as the 
current driving state. In the method proposed in this study, 
€, is set to 0.2 and g, is set to 0.6. 


4. Experiments 


4.1. Data Collection. We carry out the experiment in real 
driving and capture the facial expression video. The experi- 
ment lasts two hours. The video resolution is 640 x 480 and 
the frame rate is 30fps. The video is divided into several 
video segments with equal length. In this paper, we set the 
length of each segment and also the time interval of driving 
state prediction by FNN to be one minute. In order to get 
the standard of state assessment, we ask three people to rate 
these video segments based on the features in Table 1 and 
the driving states are confirmed if scores by three raters are 
the same. If raters have different opinions, the video needs 
to be reassessed. We extract 30 frames at equal intervals in 
every 10 seconds as test samples and size of all the frames is 
adjusted to 437 x 437. Fatigue features are extracted based on 
the methods in Section 2. The state evaluations by different 
features are obtained using FNN. In one video segment, 
BPAs are determined by normalizing the output of FNNs and 
the recognition result is obtained by the multifeature fusion 
framework based on Section 3. The process of videos and 
frames extraction is shown in Figure 11. 

Thus, we can compare the driver state assessment perfor- 
mance among the recognition results of multifeature fusion, 
single feature, and the assessment standard based on scores 
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FIGURE 11: Process of videos and frames extraction. 





FIGURE 12: Frames of fatigue recognition. 


TABLE 5: Fatigue evidence. 


Awake Fatigue Severe fatigue 
A 0.1667 0.6666 0.1667 
e, 0.8334 0.0833 0.0833 
e 0.1667 0.7500 0.0833 

TABLE 6: Fusion result. 

Awake Fatigue Severe fatigue Unknown 
eje 0.6663 0.2661 0.0666 0 
eese, 0.3502 0.6293 0.0175 0 


of video segments. The mouth movement feature may be 
affected by driver's talking, so we choose fatigue reflected in 
eyes as the single feature for comparison. The eyes feature 
is obtained by the combination of eyelid movement and iris 
movement parameters. 


4.2. Results and Analysis. In two hours of driving experiment, 
we acquire totally 120 video segments. There are 92 segments 
showing “Awake,” 21 segments showing “Fatigue,” and 7 
segments showing “Severe fatigue.” In one segment where 
the driving state is considered as “Fatigue,” several frames of 
fatigue recognition are shown in Figure 12. 

In measuring PERCLOS, MCD, AAI, PRPT, YF, and AOT, 
the width of time windows is set as 10s. The values of eyelid 
movement, iris movement, and mouth movement features are 


Driving states (score) 





Time (min) 


—~e- Real states —~a- Multifeature fusion 


-«-— Single feature 


FIGURE 13: Recognition result. 


the input of their corresponding FNN. According to formula 
(6), three BPAs of fatigue evidence are shown in Table 5, 
where e, represents the eyelid feature, e, represents the iris 
feature, and e, represents the mouth feature. 

The fusion result of DST with revision of evidence is 
shown in Table 6. 

From Tables 5 and 6, according to the recognition rule 
in Section 3.4, the driving state can be identified as “Awake” 
based on single feature, while based on multifeature fusion, 
the state is identified as “Fatigue.” 

For all the video segments, the experiment result is shown 
in Figure 13 and Table 7. Figure 13 indicates the changing 
process of driving states recognized by multifeature fusion 
and single feature. 
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TABLE 7: Comparison of different recognition results. 


Multifeature fusion 


Total segments Recognized Correct Correct 
segments segments rate 
Awake 92 93 88 95.65% 
Fatigue 21 22 15 71.43% 
Severe fatigue 7 5 4 57.14% 


The comparison of two different methods is shown in 
Table 7. The correct rate by multifeature fusion recognition 
is increased comparing to single feature recognition. The 
accuracy of fatigue driving recognition is improved using 
both eye and mouth features based on the multifeature fusion 
framework. 


5. Conclusions 


This study proposed a method for recognizing fatigue driving 
recognition. This method is based on FNN and DST to 
address the complexity of fatigue information. The BPAs 
of multiple pieces of evidences based on different visual 
features are obtained by FNNs. DST is applied for fusion 
of evidence. A modified algorithm with discount strategy is 
also used for the revision of conflicting evidence to enhance 
the rationality of fusion results. This algorithm adopts the 
correlation coefficient of evidence to measure the degree of 
conflict. The credibility of evidence, which is measured by 
the correlation coefficient, is represented as discount factor 
for evidence revision. The results of simulations indicate that 
this recognition method can overcome the interference of 
unreliable information that originated from environmental 
interferences and measurement errors. Therefore, the pro- 
posed method can increase the accuracy and robustness of 
fatigue driving recognition. 


Competing Interests 


The authors declare that there is no conflict of interests 
regarding the publication of this paper. 


Acknowledgments 


This work was supported by the Major Project of Natural 
Science by Education Department of Anhui Province (no. 
KJ2014ZD04), the Key Project of Natural Science by Educa- 
tion Department of Anhui Province (no. KJ2015A316), the 
Outstanding Young Talents at Home Visit the School Training 
Project (no. gxfxZD2016101), and National Natural Science 
Foundation of China (51605464). 


References 


[1] H.-B. Kang, “Various approaches for driver and driving behav- 
ior monitoring: a review, in Proceedings of the IEEE Interna- 
tional Conference on Computer Vision Workshops (ICCVW °13), 
pp. 616-623, IEEE, Sydney, Australia, December 2013. 

[2] C.-H. Chuang, C.-S. Huang, L.-W. Ko, and C.-T. Lin, “An EEG- 
based perceptual function integration network for application 


Single feature 


Total Recognized Correct Correct Total 
correct rate segments segments rate correct rate 
993 83 90:22% 
89.16% 23 9 40.91% 79.16% 
4 3 42.86% 


to drowsy driving, Knowledge-Based Systems, vol. 80, pp. 143- 
152, 2015. 


[3] R. Fu and H. Wang, “Detection of driving fatigue by using 
noncontact emg and ecg signals measurement system,” Inter- 
national Journal of Neural Systems, vol. 24, no. 3, Article ID 
1450006, 15 pages, 2014. 


[4] M. Patel, S. K. L. Lal, D. Kavanagh, and P. Rossiter, “Applying 
neural network analysis on heart rate variability data to assess 
driver fatigue, Expert Systems with Applications, vol. 38, no. 6, 
pp. 7235-7242, 2011. 

[5] S. Ahn, T. Nguyen, H. Jang, J. G. Kim, and S. C. Jun, “Exploring 
neuro-physiological correlates of drivers mental fatigue caused 
by sleep deprivation using simultaneous EEG, ECG, and f NIRS 
data,’ Frontiers in Human Neuroscience, vol. 10, no. 2016, article 
219, 2016. 


[6] L. M. Bergasa, J. Nuevo, M. A. Sotelo, R. Barea, and M. E. 
Lopez, “Real-time system for monitoring driver vigilance, IEEE 
Transactions on Intelligent Transportation Systems, vol. 7, no. 1, 
pp. 63-77, 2006. 


[7] T. D'Orazio, M. Leo, C. Guaragnella, and A. Distante, “A visual 
approach for driver inattention detection,” Pattern Recognition, 
vol. 40, no. 8, pp. 2341-2355, 2007. 


[8] M. Zhang, G. Longhui, Z. Wang, X. Xu, B. Yao, and L. Zhou, 
“Hybrid model for early onset prediction of driver fatigue with 
observable cues? Mathematical Problems in Engineering, vol. 
2014, Article ID 385716, 9 pages, 2014. 


[9] J. Jimenez-Pinto and M. Torres-Torriti, “Face salient points and 
eyes tracking for robust drowsiness detection,” Robotica, vol. 30, 
no. 5, pp. 731-741, 2012. 


[10] W. Sun, X. Zhang, J. Wang, J. He, and S. Peeta, “Blink number 
forecasting based on improved bayesian fusion algorithm for 
fatigue driving detection, Mathematical Problems in Engineer- 
ing, vol. 2015, Article ID 832621, 13 pages, 2015. 


[11] Y. Du, Q. Hu, D. Chen, and P. Ma, “Kernelized fuzzy rough 
sets based yawn detection for driver fatigue monitoring, 
Fundamenta Informaticae, vol. 111, no. 1, pp. 65-79, 2011. 


[12] J. Son, H. Yoo, S. Kim, and K. Sohn, “Real-time illumination 
invariant lane detection for lane departure warning system,” 
Expert Systems with Applications, vol. 42, no. 4, pp. 1816-1824, 
2015. 


[13] S.-J. Jung, H.-S. Shin, and W.-Y. Chung, “Driver fatigue and 
drowsiness monitoring system with embedded electrocardio- 
gram sensor on steering wheel,’ IET Intelligent Transport Sys- 
tems, vol. 8, no. 1, pp. 43-50, 2014. 


[14] A. Sahayadhas, K. Sundaraj, and M. Murugappan, “Detecting 
driver drowsiness based on sensors: a review, Sensors, vol. 12, 
no. 12, pp. 16937-16953, 2012. 

[15] B.-G. Lee and W.-Y. Chung, “Driver alertness monitoring using 


fusion of facial features and bio-signals,’ IEEE Sensors Journal, 
vol. 12, no. 7, pp. 2416-2422, 2012. 


10 


[16] Y. Liang, M. L. Reyes, and J. D. Lee, “Real-time detection of 
driver cognitive distraction using support vector machines,’ 
IEEE Transactions on Intelligent Transportation Systems, vol. 8, 
no. 2, pp. 340-350, 2007. 

[17] S. Deng, X. Xu, X. Yang, and D. Miao, “Research on the driver 
fatigue monitoring method based on the Dempster-Shafer 
theory, in Proceedings of the Chinese Control and Decision 
Conference (CCDC ’10), pp. 4176-4179, IEEE, Xuzhou, China, 
May 2010. 

[18] A. P. Dempster, “Upper and lower probabilities induced by a 
multivalued mapping,’ The Annals of Mathematical Statistics, 
vol. 38, no. 2, pp. 325-339, 1967. 

[19] G. Shafer, A Mathematical Theory of Evidence, vol. 1, Princeton 
University Press, Princeton, NJ, USA, 1976. 

[20] L. A. Zadeh, “Simple view of the dempster-shafer theory of 
evidence and its implication for the rule of combination, AI 
Magazine, vol. 7, no. 2, pp. 85-90, 1986. 

[21] Y. Deng, “Generalized evidence theory,’ Applied Intelligence, vol. 
43, no. 3, pp. 530-543, 2015. 

[22] R. R. Yager, “On the dempster-shafer framework and new 
combination rules,’ Information Sciences, vol. 41, no. 2, pp. 93- 
137, 1987. 

[23] S. Y. Qian, Y. Xu, and Y. Gu, “Combination of spin-trapping, 
LC/ESR and LC/MS technique in characterization of PUFA- 
derived free radicals in lipid peroxidation,’ Acta Biophysica 
Sinica, vol. 28, no. 5, p. 355, 2012. 

[24] E. Lefevre, O. Colot, and P. Vannoorenberghe, “Belief function 
combination and conflict management,’ Information Fusion, 
vol. 3, no. 2, pp. 149-162, 2002. 

[25] E Smarandache and J. Dezert, Advances and Applications 
of DSmT for Information Fusion, vol. 2 of Collected Works, 
American Research Press, 2006. 

[26] C. K. Murphy, “Combining belief functions when evidence 
conflicts,’ Decision Support Systems, vol. 29, no. 1, pp. 1-9, 2000. 

[27] D. Yong, S. WenKang, Z. ZhenFu, and L. Qi, “Combining belief 
functions based on distance of evidence,’ Decision Support 
Systems, vol. 38, no. 3, pp. 489-493, 2004. 

[28] P. Smets, “Decision making in the TBM: the necessity of the 
pignistic transformation, International Journal of Approximate 
Reasoning, vol. 38, no. 2, pp. 133-147, 2005. 

[29] W. Liu, “Analyzing the degree of conflict among belief func- 
tions,” Artificial Intelligence, vol. 170, no. 11, pp. 909-924, 2006. 

[30] J. Schubert, “Conflict management in Dempster-Shafer theory 
using the degree of falsity,’ International Journal of Approximate 
Reasoning, vol. 52, no. 3, pp. 449-460, 2011. 

[31] M. Li, X. Lu, Q. Zhang, and Y. Deng, “Multiscale probability 
transformation of basic probability assignment,’ Mathematical 
Problems in Engineering, vol. 2014, Article ID 319264, 6 pages, 
2014. 

[32] W. Jiang, C. Xie, M. Zhuang, Y. Shou, and Y. Tang, “Sensor data 
fusion with z-numbers and its application in fault diagnosis,’ 
Sensors, vol. 16, no. 9, p. 1509, 2016. 

[33] L. A. Zadeh, “A Note on Z-numbers,’ Information Sciences, vol. 
181, no. 14, pp. 2923-2932, 2011. 

[34] P Smets and R. Kennes, “The transferable belief model,” 
Artificial Intelligence, vol. 66, no. 2, pp. 191-234, 1994. 

[35] W. Jiang, B. Wei, C. Xie, and D. Zhou, “An evidential sensor 
fusion method in fault diagnosis? Advances in Mechanical 
Engineering, vol. 8, no. 3, pp. 1-7, 2016. 

[36] Y. Deng, “Deng entropy,’ Chaos, Solitons & Fractals, vol. 91, pp. 
549-553, 2016. 


Mathematical Problems in Engineering 


[37] R. Babuška and H. Verbruggen, “Neuro-fuzzy methods for 
nonlinear system identification,’ Annual Reviews in Control, vol. 
27, pp. 73-85, 2003. 


[38] Y.-C. Chen and C.-C. Teng, “A model reference control struc- 
ture using a fuzzy neural network,’ Fuzzy Sets and Systems, vol. 
73, 110295 Pp» 291=312; 1995. 


[39] T. Kimura, K. Ishida, and N. Ozaki, “Feasibility study of sleepi- 
ness detection using expression features,’ Review of Automotive 
Engineering, vol. 29, no. 4, pp. 567-574, 2008. 


[40] D. Cristinacce and T. F. Cootes, “Feature detection and tracking 
with constrained local models,’ in Proceedings of the British 
Machine Vision Conference, vol. 1, p. 3, 2006. 


[41] W. W. Wierwille, S. S. Wreggit, C. L. Kirn, L. A. Ellsworth, 
and R. J. Fairbanks, “Research on vehicle-based driver sta- 
tus/performance monitoring; development, validation, and 
refinement of algorithms for detection of driver drowsiness,” 
Final Report HS-808 247, National Highway Trafhc Safety 
Administration, 1994. 


[42] J. Canny, “A computational approach to edge detection,” IEEE 
Transactions on Pattern Analysis and Machine Intelligence, vol. 
8, no. 6, pp. 679-698, 1986. 


[43] V. C. Hough, “Method and means for recognizing complex 
patterns,’ U.S. Patent no. 3,069,654, 1962. 


[44] Y. Zhao, J. Li, L. Li, M. Zhang, and L. Guo, “Environmental 
perception and sensor data fusion for unmanned ground vehi- 
cle,’ Mathematical Problems in Engineering, vol. 2013, Article ID 
903951, 12 pages, 2013. 


[45] Z. Dou, X. Xu, Y. Lin, and R. Zhou, “Application of D-S evidence 
fusion method in the fault detection of temperature sensor,’ 
Mathematical Problems in Engineering, vol. 2014, Article ID 
395057, 6 pages, 2014. 


[46] W. Jiang, J. Zhan, D. Zhou, and X. Li, “A method to determine 
generalized basic probability assignment in the open world, 
Mathematical Problems in Engineering, vol. 2016, Article ID 
3878634, 11 pages, 2016. 


[47] X. Guan, X. Yi, and Y. He, “Study on algorithms of determining 
basic probability assignment function in Dempster-Shafer evi- 
dence theory,’ in Proceedings of the 7th International Conference 
on Machine Learning and Cybernetics (ICMLC °08), pp. 121-126, 
July 2008. 


